PowerNet: Multi-Agent Deep Reinforcement Learning for Scalable Powergrid Control

This paper develops an efficient multi-agent deep reinforcement learning algorithm for cooperative controls in powergrids. Specifically, we consider the decentralized inverter-based secondary voltage control problem distributed generators (DGs), which is first formulated as a (MARL) problem. We then propose novel on-policy MARL algorithm, PowerNet, each agent (DG) learns policy based on (sub-)global reward but local states and encoded communication messages from its neighbors. Motivated by fact that one has limited impact agents distant it, exploit spatial discount factor to reduce effect remote agents, expedite training process improve scalability. Furthermore, differentiable, learning-based protocol employed foster collaborations among neighboring agents. In addition, mitigate effects of system uncertainty random noise introduced during learning, utilize action smoothing stabilize execution. To facilitate evaluation, develop PGSim, efficient, high-fidelity powergrid simulation platform. Experimental results two microgrid setups show developed PowerNet outperforms conventional model-based method, well several state-of-the-art algorithms. The scheme high sample efficiency also make it viable large-scale power grids.